Trasfer learning: Food Vision (Food101)¶

In [1]:
!nvidia-smi
Mon Mar 13 22:02:51 2023       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 530.30.02              Driver Version: 531.18       CUDA Version: 12.1     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                  Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf            Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3060 L...    On | 00000000:01:00.0 Off |                  N/A |
| N/A   52C    P8               10W /  N/A|     11MiB /  6144MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A        17      G   /Xwayland                                 N/A      |
|    0   N/A  N/A       126      G   /Xwayland                                 N/A      |
|    0   N/A  N/A       618      G   /Xwayland                                 N/A      |
|    0   N/A  N/A     10403      G   /Xwayland                                 N/A      |
+---------------------------------------------------------------------------------------+
In [2]:
import tensorflow as tf
2023-03-13 22:02:52.143172: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
In [3]:
# Import helper functions
from helper_functions import create_tensorboard_callback, plot_loss_curves, unzip_data, walk_through_dir, compare_historys

101 Food Classes: working with less data¶

In [4]:
!wget https://storage.googleapis.com/ztm_tf_course/food_vision/101_food_classes_10_percent.zip
--2023-03-13 22:02:56--  https://storage.googleapis.com/ztm_tf_course/food_vision/101_food_classes_10_percent.zip
Resolving storage.googleapis.com (storage.googleapis.com)... 142.251.41.48, 142.251.41.80, 172.217.1.16, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|142.251.41.48|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1625420029 (1.5G) [application/zip]
Saving to: ‘101_food_classes_10_percent.zip.1’

101_food_classes_10 100%[===================>]   1.51G  54.5MB/s    in 29s     

2023-03-13 22:03:26 (52.9 MB/s) - ‘101_food_classes_10_percent.zip.1’ saved [1625420029/1625420029]

In [5]:
unzip_data("101_food_classes_10_percent.zip")
In [6]:
train_dir = "101_food_classes_10_percent/train"
test_dir = "101_food_classes_10_percent/test"
In [7]:
# Check directory
walk_through_dir("101_food_classes_10_percent")
There are 2 directories and 0 images in '101_food_classes_10_percent'.
There are 101 directories and 0 images in '101_food_classes_10_percent/train'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/frozen_yogurt'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/pad_thai'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/crab_cakes'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/french_toast'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/cheese_plate'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/churros'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/pancakes'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/beef_carpaccio'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/escargots'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/tuna_tartare'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/dumplings'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/chocolate_cake'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/baby_back_ribs'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/fried_rice'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/pork_chop'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/shrimp_and_grits'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/fish_and_chips'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/eggs_benedict'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/foie_gras'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/chicken_wings'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/beignets'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/ramen'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/waffles'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/caesar_salad'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/breakfast_burrito'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/filet_mignon'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/lobster_bisque'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/panna_cotta'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/nachos'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/garlic_bread'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/deviled_eggs'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/guacamole'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/french_onion_soup'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/poutine'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/huevos_rancheros'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/bread_pudding'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/pulled_pork_sandwich'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/baklava'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/gyoza'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/club_sandwich'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/onion_rings'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/chocolate_mousse'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/miso_soup'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/steak'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/samosa'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/peking_duck'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/hot_and_sour_soup'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/ice_cream'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/pizza'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/oysters'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/fried_calamari'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/scallops'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/hamburger'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/seaweed_salad'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/spring_rolls'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/lasagna'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/takoyaki'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/donuts'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/cannoli'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/sashimi'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/grilled_salmon'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/cup_cakes'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/bruschetta'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/ceviche'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/creme_brulee'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/spaghetti_bolognese'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/beef_tartare'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/chicken_curry'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/macaroni_and_cheese'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/prime_rib'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/ravioli'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/carrot_cake'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/macarons'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/paella'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/gnocchi'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/sushi'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/hot_dog'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/tacos'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/clam_chowder'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/chicken_quesadilla'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/risotto'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/edamame'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/cheesecake'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/greek_salad'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/red_velvet_cake'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/caprese_salad'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/grilled_cheese_sandwich'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/falafel'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/apple_pie'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/beet_salad'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/mussels'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/bibimbap'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/spaghetti_carbonara'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/omelette'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/strawberry_shortcake'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/french_fries'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/tiramisu'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/lobster_roll_sandwich'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/hummus'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/pho'.
There are 0 directories and 75 images in '101_food_classes_10_percent/train/croque_madame'.
There are 101 directories and 0 images in '101_food_classes_10_percent/test'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/frozen_yogurt'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/pad_thai'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/crab_cakes'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/french_toast'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/cheese_plate'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/churros'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/pancakes'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/beef_carpaccio'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/escargots'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/tuna_tartare'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/dumplings'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/chocolate_cake'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/baby_back_ribs'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/fried_rice'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/pork_chop'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/shrimp_and_grits'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/fish_and_chips'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/eggs_benedict'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/foie_gras'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/chicken_wings'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/beignets'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/ramen'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/waffles'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/caesar_salad'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/breakfast_burrito'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/filet_mignon'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/lobster_bisque'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/panna_cotta'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/nachos'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/garlic_bread'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/deviled_eggs'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/guacamole'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/french_onion_soup'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/poutine'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/huevos_rancheros'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/bread_pudding'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/pulled_pork_sandwich'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/baklava'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/gyoza'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/club_sandwich'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/onion_rings'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/chocolate_mousse'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/miso_soup'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/steak'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/samosa'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/peking_duck'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/hot_and_sour_soup'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/ice_cream'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/pizza'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/oysters'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/fried_calamari'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/scallops'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/hamburger'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/seaweed_salad'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/spring_rolls'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/lasagna'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/takoyaki'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/donuts'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/cannoli'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/sashimi'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/grilled_salmon'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/cup_cakes'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/bruschetta'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/ceviche'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/creme_brulee'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/spaghetti_bolognese'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/beef_tartare'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/chicken_curry'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/macaroni_and_cheese'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/prime_rib'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/ravioli'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/carrot_cake'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/macarons'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/paella'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/gnocchi'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/sushi'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/hot_dog'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/tacos'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/clam_chowder'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/chicken_quesadilla'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/risotto'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/edamame'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/cheesecake'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/greek_salad'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/red_velvet_cake'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/caprese_salad'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/grilled_cheese_sandwich'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/falafel'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/apple_pie'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/beet_salad'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/mussels'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/bibimbap'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/spaghetti_carbonara'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/omelette'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/strawberry_shortcake'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/french_fries'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/tiramisu'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/lobster_roll_sandwich'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/hummus'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/pho'.
There are 0 directories and 250 images in '101_food_classes_10_percent/test/croque_madame'.
In [8]:
# Setup data inputs
import tensorflow as tf

IMG_SIZE = (224, 224)

train_data_all_10_percent = tf.keras.preprocessing.image_dataset_from_directory(train_dir,
                                                                               image_size=IMG_SIZE,
                                                                               label_mode="categorical")

test_data = tf.keras.preprocessing.image_dataset_from_directory(test_dir,
                                                               image_size=IMG_SIZE,
                                                               label_mode="categorical",
                                                               shuffle=False)
Found 7575 files belonging to 101 classes.
2023-03-13 22:03:43.809282: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2023-03-13 22:03:43.955861: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2023-03-13 22:03:43.955913: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2023-03-13 22:03:43.959364: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-03-13 22:03:43.964089: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2023-03-13 22:03:43.964189: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2023-03-13 22:03:43.964215: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2023-03-13 22:03:46.940174: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2023-03-13 22:03:46.940272: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2023-03-13 22:03:46.940281: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2023-03-13 22:03:46.940312: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:961] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2023-03-13 22:03:46.940374: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3433 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3060 Laptop GPU, pci bus id: 0000:01:00.0, compute capability: 8.6
Found 25250 files belonging to 101 classes.

Train a big model with transfer learning on 10% of 101 food classes¶

Steps:

  • Create a ModelCheckpoint callback
  • Create a data augmentation layer
  • Build a headless (no top layers) Functional EfficientNetB0 backboned-model
  • Compile our model
  • Feature extract for full passes
In [9]:
# Creat checkpoint callback
checkpoint_path = "101_classes_10_percent_data_model_checkpoint/checkpoints.ckpt"
checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_path,
                                                        save_weights_only=True,
                                                        monitor="val_accuracy",
                                                        save_best_only=True)
In [10]:
# Create data augmentation layer
from tensorflow.keras import layers
from tensorflow.keras import Sequential

data_augmentation = Sequential([
    layers.RandomFlip("horizontal"),
    layers.RandomRotation(0.2),
    layers.RandomHeight(0.2),
    layers.RandomWidth(0.2),
    layers.RandomZoom(0.2),
    # layers.Rescaling(1/255.) # Rescale images required for ResNet50v2
], name="data_augmentation")
In [11]:
# Setup the base mode and freeze its layers (to extract features)
base_model = tf.keras.applications.EfficientNetB0(include_top=False)
base_model.trainable = False

# Setup model architecture with trainable top layres
inputs = layers.Input(shape=(224, 224, 3), name="input_layer")
x = data_augmentation(inputs) # Augment images during training
x = base_model(x, training=False) # put base model in inference mode to set weights frozen
x = layers.GlobalAveragePooling2D(name="global_avg_pool_layer")(x)
outputs = layers.Dense(len(train_data_all_10_percent.class_names), activation="softmax", name="output_layer")(x)

model = tf.keras.Model(inputs, outputs)
In [12]:
# Compile the model
model.compile(loss="categorical_crossentropy",
             optimizer=tf.keras.optimizers.Adam(),
             metrics=["accuracy"])
In [13]:
model.summary()
Model: "model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_layer (InputLayer)    [(None, 224, 224, 3)]     0         
                                                                 
 data_augmentation (Sequenti  (None, 224, 224, 3)      0         
 al)                                                             
                                                                 
 efficientnetb0 (Functional)  (None, None, None, 1280)  4049571  
                                                                 
 global_avg_pool_layer (Glob  (None, 1280)             0         
 alAveragePooling2D)                                             
                                                                 
 output_layer (Dense)        (None, 101)               129381    
                                                                 
=================================================================
Total params: 4,178,952
Trainable params: 129,381
Non-trainable params: 4,049,571
_________________________________________________________________
In [14]:
# Fit the model
base_model_history_101_food_classes_10_percent = model.fit(train_data_all_10_percent,
                                                          epochs=5,
                                                          validation_data=test_data,
                                                          validation_steps=int(0.15*len(test_data)),
                                                          callbacks=[checkpoint_callback, create_tensorboard_callback(dir_name="transfer_learning",
                                                                                                                       experiment_name="base_model_history_101_food_classes_10_percent")])
Saving TensorBoard log files to: transfer_learning/base_model_history_101_food_classes_10_percent/20230313-220351
Epoch 1/5
2023-03-13 22:04:02.288733: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8700
  1/237 [..............................] - ETA: 58:45 - loss: 4.7971 - accuracy: 0.0000e+00
2023-03-13 22:04:06.693938: I tensorflow/stream_executor/cuda/cuda_blas.cc:1786] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
237/237 [==============================] - 80s 278ms/step - loss: 3.3773 - accuracy: 0.2750 - val_loss: 2.4317 - val_accuracy: 0.4828
Epoch 2/5
237/237 [==============================] - 54s 229ms/step - loss: 2.2031 - accuracy: 0.5002 - val_loss: 2.0229 - val_accuracy: 0.5204
Epoch 3/5
237/237 [==============================] - 51s 217ms/step - loss: 1.8199 - accuracy: 0.5642 - val_loss: 1.8875 - val_accuracy: 0.5334
Epoch 4/5
237/237 [==============================] - 49s 208ms/step - loss: 1.6030 - accuracy: 0.6141 - val_loss: 1.7667 - val_accuracy: 0.5564
Epoch 5/5
237/237 [==============================] - 48s 205ms/step - loss: 1.4552 - accuracy: 0.6430 - val_loss: 1.7681 - val_accuracy: 0.5501
In [15]:
# Evaluate on whole test dataset
feature_extraction_results = model.evaluate(test_data)
feature_extraction_results
790/790 [==============================] - 49s 61ms/step - loss: 1.5844 - accuracy: 0.5837
Out[15]:
[1.5844230651855469, 0.58368319272995]
In [74]:
plot_loss_curves(base_model_history_101_food_classes_10_percent)

Fine tuning¶

In [23]:
# unfreeze all of the layers in the base model
base_model.trainable = True

# Refreeze every layer except last 5
for layer in base_model.layers[:-5]:
    layer.trainable = False
In [24]:
for i, layer in enumerate(base_model.layers[-10:]):
    print(i, layer.name, layer.trainable)
    
0 block7a_se_squeeze False
1 block7a_se_reshape False
2 block7a_se_reduce False
3 block7a_se_expand False
4 block7a_se_excite False
5 block7a_project_conv True
6 block7a_project_bn True
7 top_conv True
8 top_bn True
9 top_activation True
In [25]:
# Recompiling model with lower learning_rate
model.compile(loss="categorical_crossentropy",
             optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001),
             metrics=["accuracy"])
In [26]:
model.summary()
Model: "model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_layer (InputLayer)    [(None, 224, 224, 3)]     0         
                                                                 
 data_augmentation (Sequenti  (None, 224, 224, 3)      0         
 al)                                                             
                                                                 
 efficientnetb0 (Functional)  (None, None, None, 1280)  4049571  
                                                                 
 global_avg_pool_layer (Glob  (None, 1280)             0         
 alAveragePooling2D)                                             
                                                                 
 output_layer (Dense)        (None, 101)               129381    
                                                                 
=================================================================
Total params: 4,178,952
Trainable params: 910,821
Non-trainable params: 3,268,131
_________________________________________________________________
In [27]:
# Fit the model again
initial_epochs = 5
history_fine_tuned_base_model_101_food_classes_10_percent = model.fit(train_data_all_10_percent,
                                                                     epochs=initial_epochs + 5,
                                                                     validation_data=test_data,
                                                                     validation_steps=int(0.15 * len(test_data)),
                                                                     initial_epoch=base_model_history_101_food_classes_10_percent.epoch[-1],
                                                                     callbacks=[create_tensorboard_callback(dir_name="transfer_learning",
                                                                                                           experiment_name="history_fine_tuned_base_model_101_food_classes_10_percent")])
Saving TensorBoard log files to: transfer_learning/history_fine_tuned_base_model_101_food_classes_10_percent/20230313-221816
Epoch 5/10
237/237 [==============================] - 47s 174ms/step - loss: 1.2255 - accuracy: 0.6796 - val_loss: 1.7484 - val_accuracy: 0.5434
Epoch 6/10
237/237 [==============================] - 39s 164ms/step - loss: 1.0948 - accuracy: 0.7083 - val_loss: 1.7466 - val_accuracy: 0.5416
Epoch 7/10
237/237 [==============================] - 39s 165ms/step - loss: 1.0149 - accuracy: 0.7291 - val_loss: 1.7797 - val_accuracy: 0.5437
Epoch 8/10
237/237 [==============================] - 40s 169ms/step - loss: 0.9560 - accuracy: 0.7442 - val_loss: 1.7639 - val_accuracy: 0.5408
Epoch 9/10
237/237 [==============================] - 38s 161ms/step - loss: 0.8910 - accuracy: 0.7603 - val_loss: 1.7392 - val_accuracy: 0.5477
Epoch 10/10
237/237 [==============================] - 37s 158ms/step - loss: 0.8374 - accuracy: 0.7674 - val_loss: 1.7335 - val_accuracy: 0.5548
In [28]:
# Evaluate on whole dataset
all_classes_10_percent_fine_tune_results = model.evaluate(test_data)
all_classes_10_percent_fine_tune_results
790/790 [==============================] - 47s 60ms/step - loss: 1.4837 - accuracy: 0.6080
Out[28]:
[1.4836808443069458, 0.6080396175384521]
In [29]:
# Compare the histories with fine tuning
compare_historys(original_history=base_model_history_101_food_classes_10_percent,
                new_history=history_fine_tuned_base_model_101_food_classes_10_percent,
                initial_epochs=5)

As we can see, after fine tuning, validation loss and accuracy stays flat while training values starts improving.

Saving and loading our model¶

to use it in extrenal applications

In [ ]:
# Save our model in h5 format
model.save("FastAPI_app/model/fine_tuned_model_v1.h5")
In [ ]:
# Load and evaluate saved model
loaded_model = tf.keras.models.load_model("FastAPI_app/model/fine_tuned_model_v1.h5")
In [ ]:
# Evaluate loaded models
loaded_model_results = loaded_model.evaluate(test_data)
loaded_model_results
790/790 [==============================] - 42s 51ms/step - loss: 1.4837 - accuracy: 0.6082
Out[ ]:
[1.4836807250976562, 0.6081584095954895]
In [ ]:
# Model results before loading
all_classes_10_percent_fine_tune_results
Out[ ]:
[1.4836808443069458, 0.6080396175384521]

Making perdictions with our trained model¶

In [ ]:
# Make predictions with model
preds_probs =  loaded_model.predict(test_data, verbose=1) # set verbosity to see how log is left
790/790 [==============================] - 45s 56ms/step
In [ ]:
# How many predictions are there
len(preds_probs)
Out[ ]:
25250
In [ ]:
# Shapes of out predictions
preds_probs.shape
Out[ ]:
(25250, 101)
In [ ]:
# let's see what first 10 predictions look like
preds_probs[:10]
Out[ ]:
array([[2.13262409e-01, 3.74424621e-04, 1.47300526e-01, ...,
        1.00088573e-05, 1.19277136e-02, 1.11554479e-02],
       [8.28700900e-01, 6.14220639e-08, 5.82025561e-04, ...,
        5.36216982e-03, 8.75272349e-07, 6.64483878e-06],
       [7.16566741e-01, 3.80486576e-03, 9.46540292e-03, ...,
        2.49304052e-04, 3.07775714e-04, 7.64966302e-04],
       ...,
       [1.75064757e-01, 4.42263627e-06, 2.12276657e-03, ...,
        2.64248643e-02, 2.05834187e-03, 1.07281754e-04],
       [1.79688051e-01, 3.12993274e-04, 3.71866375e-01, ...,
        1.75711684e-04, 1.91263462e-04, 1.12177455e-04],
       [6.95914507e-01, 5.97044789e-07, 1.16489735e-03, ...,
        5.06197801e-03, 2.16848321e-06, 4.51417465e-04]], dtype=float32)
In [ ]:
preds_probs[0], max(preds_probs[0]), sum(preds_probs[0])
Out[ ]:
(array([2.13262409e-01, 3.74424621e-04, 1.47300526e-01, 1.59269959e-07,
        5.91850358e-06, 3.17156633e-07, 1.20911223e-04, 8.32445585e-06,
        2.35058926e-02, 4.98985522e-04, 5.41419979e-07, 5.36608877e-06,
        4.99380636e-04, 3.42688111e-07, 9.79942549e-03, 1.10681122e-03,
        7.09556889e-06, 1.90289458e-03, 4.27856995e-03, 6.41824299e-05,
        1.63555983e-03, 1.23004947e-05, 3.44534026e-04, 9.38770390e-05,
        1.00595318e-03, 4.63977084e-03, 1.50340248e-03, 1.49832063e-04,
        1.44488811e-01, 4.75591660e-04, 3.27649764e-06, 1.08300371e-03,
        1.51802588e-03, 6.21370077e-07, 2.83269398e-02, 4.24230302e-06,
        2.03907286e-04, 1.30680180e-03, 1.61797553e-02, 4.50966145e-05,
        7.23947538e-04, 5.20048104e-03, 4.85921395e-04, 1.48963227e-05,
        1.27142383e-04, 8.61562603e-06, 2.52367766e-03, 1.21461235e-04,
        2.39577480e-06, 6.96821660e-02, 1.67483566e-04, 2.01781499e-04,
        7.40272775e-02, 2.39406526e-02, 8.61716253e-05, 7.94744957e-03,
        3.22495587e-02, 1.27961896e-02, 9.06321965e-03, 1.05639199e-04,
        4.49018188e-07, 2.03054701e-03, 3.93103398e-02, 8.40474793e-04,
        5.66966308e-04, 1.13091191e-04, 4.20528704e-05, 4.35997918e-03,
        1.66725868e-03, 3.07308801e-04, 4.17302304e-04, 8.79140407e-06,
        1.87891629e-03, 3.62995088e-05, 7.49920844e-04, 1.67468515e-05,
        1.35470065e-04, 7.19289575e-03, 2.10717510e-04, 2.20792354e-04,
        6.01738750e-04, 2.11758597e-04, 5.92641416e-04, 3.09142997e-05,
        5.03251737e-04, 6.11881167e-02, 2.93529755e-03, 8.87527072e-04,
        1.05828385e-05, 2.35582484e-04, 5.88492503e-06, 2.00948434e-05,
        8.81854794e-04, 7.05208658e-05, 8.66177870e-05, 9.61464015e-04,
        1.99602032e-03, 3.40731902e-04, 1.00088573e-05, 1.19277136e-02,
        1.11554479e-02], dtype=float32),
 0.21326241,
 1.0000000030027394)
In [ ]:
# we get one prediction probability per class
print(f"Number of prediction probabilities for sample 0: {len(preds_probs[0])}")
print(f"Waht prediction probability sample 0 looks like:\n {preds_probs[0]}")
print(f"The class with the heighest predicted probability by the model for sample 0: {preds_probs[0].argmax()}")
Number of prediction probabilities for sample 0: 101
Waht prediction probability sample 0 looks like:
 [2.13262409e-01 3.74424621e-04 1.47300526e-01 1.59269959e-07
 5.91850358e-06 3.17156633e-07 1.20911223e-04 8.32445585e-06
 2.35058926e-02 4.98985522e-04 5.41419979e-07 5.36608877e-06
 4.99380636e-04 3.42688111e-07 9.79942549e-03 1.10681122e-03
 7.09556889e-06 1.90289458e-03 4.27856995e-03 6.41824299e-05
 1.63555983e-03 1.23004947e-05 3.44534026e-04 9.38770390e-05
 1.00595318e-03 4.63977084e-03 1.50340248e-03 1.49832063e-04
 1.44488811e-01 4.75591660e-04 3.27649764e-06 1.08300371e-03
 1.51802588e-03 6.21370077e-07 2.83269398e-02 4.24230302e-06
 2.03907286e-04 1.30680180e-03 1.61797553e-02 4.50966145e-05
 7.23947538e-04 5.20048104e-03 4.85921395e-04 1.48963227e-05
 1.27142383e-04 8.61562603e-06 2.52367766e-03 1.21461235e-04
 2.39577480e-06 6.96821660e-02 1.67483566e-04 2.01781499e-04
 7.40272775e-02 2.39406526e-02 8.61716253e-05 7.94744957e-03
 3.22495587e-02 1.27961896e-02 9.06321965e-03 1.05639199e-04
 4.49018188e-07 2.03054701e-03 3.93103398e-02 8.40474793e-04
 5.66966308e-04 1.13091191e-04 4.20528704e-05 4.35997918e-03
 1.66725868e-03 3.07308801e-04 4.17302304e-04 8.79140407e-06
 1.87891629e-03 3.62995088e-05 7.49920844e-04 1.67468515e-05
 1.35470065e-04 7.19289575e-03 2.10717510e-04 2.20792354e-04
 6.01738750e-04 2.11758597e-04 5.92641416e-04 3.09142997e-05
 5.03251737e-04 6.11881167e-02 2.93529755e-03 8.87527072e-04
 1.05828385e-05 2.35582484e-04 5.88492503e-06 2.00948434e-05
 8.81854794e-04 7.05208658e-05 8.66177870e-05 9.61464015e-04
 1.99602032e-03 3.40731902e-04 1.00088573e-05 1.19277136e-02
 1.11554479e-02]
The class with the heighest predicted probability by the model for sample 0: 0
In [ ]:
# Get the pred classes of each label
pred_classes = preds_probs.argmax(axis=1)

pred_classes[:10]
Out[ ]:
array([ 0,  0,  0,  8,  8, 78, 29, 46,  2,  0])
In [ ]:
len(pred_classes)
Out[ ]:
25250

Now, compate pred_classes to test_data labels

In [ ]:
# To get out test labels we need to unravel out test_data BatchDataset
y_label = []
for images, labels in test_data.unbatch():
    y_label.append(labels.numpy().argmax()) # currently test labels look like: [0, 0, 0, 0, 1 ... , 0, 0]
    
y_label[:10] # look at first 10
Out[ ]:
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
In [ ]:
len(y_label)
Out[ ]:
25250

Evaluating our model's predictions¶

In [ ]:
loaded_model_results
Out[ ]:
[1.4836807250976562, 0.6081584095954895]
In [ ]:
# trying scikit-learn's accuracy score function
from sklearn.metrics import accuracy_score
sklearn_accuracy = accuracy_score(y_true=y_label, y_pred=pred_classes)
sklearn_accuracy
Out[ ]:
0.6081584158415841

Making confusion matrix¶

In [ ]:
from helper_functions import make_confusion_matrix
In [ ]:
import itertools
import matplotlib.pyplot as plt
import numpy as np
from sklearn.metrics import confusion_matrix

#Need to make some changes in confusion matrix
def make_confusion_matrix(y_true, y_pred, classes=None, figsize=(10, 10), text_size=15, norm=False, savefig=False):
    """Makes a labelled confusion matrix comparing predictions and ground truth labels.

    If classes is passed, confusion matrix will be labelled, if not, integer class values
    will be used.

    Args:
      y_true: Array of truth labels (must be same shape as y_pred).
      y_pred: Array of predicted labels (must be same shape as y_true).
      classes: Array of class labels (e.g. string form). If `None`, integer labels are used.
      figsize: Size of output figure (default=(10, 10)).
      text_size: Size of output figure text (default=15).
      norm: normalize values or not (default=False).
      savefig: save confusion matrix to file (default=False).

    Returns:
      A labelled confusion matrix plot comparing y_true and y_pred.

    Example usage:
      make_confusion_matrix(y_true=test_labels, # ground truth test labels
                            y_pred=y_preds, # predicted labels
                            classes=class_names, # array of class label names
                            figsize=(15, 15),
                            text_size=10)
    """
    # Create the confustion matrix
    cm = confusion_matrix(y_true, y_pred)
    cm_norm = cm.astype("float") / cm.sum(axis=1)[:, np.newaxis]  # normalize it
    n_classes = cm.shape[0]  # find the number of classes we're dealing with

    # Plot the figure and make it pretty
    fig, ax = plt.subplots(figsize=figsize)
    cax = ax.matshow(cm, cmap=plt.cm.Blues)  # colors will represent how 'correct' a class is, darker == better
    fig.colorbar(cax)

    # Are there a list of classes?
    if classes:
        labels = classes
    else:
        labels = np.arange(cm.shape[0])

    # Label the axes
    ax.set(title="Confusion Matrix",
           xlabel="Predicted label",
           ylabel="True label",
           xticks=np.arange(n_classes),  # create enough axis slots for each class
           yticks=np.arange(n_classes),
           xticklabels=labels,  # axes will labeled with class names (if they exist) or ints
           yticklabels=labels)

    # Make x-axis labels appear on bottom
    ax.xaxis.set_label_position("bottom")
    ax.xaxis.tick_bottom()
    
    # plotting x labels vertically
    plt.xticks(rotation=70, fontsize=text_size)

    # Set the threshold for different colors
    threshold = (cm.max() + cm.min()) / 2.

    # Plot the text on each cell
    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
        if norm:
            plt.text(j, i, f"{cm[i, j]} ({cm_norm[i, j] * 100:.1f}%)",
                     horizontalalignment="center",
                     color="white" if cm[i, j] > threshold else "black",
                     size=text_size)
        else:
            plt.text(j, i, f"{cm[i, j]}",
                     horizontalalignment="center",
                     color="white" if cm[i, j] > threshold else "black",
                     size=text_size)

    # Save the figure to the current working directory
    if savefig:
        fig.savefig("confusion_matrix.png")
In [ ]:
make_confusion_matrix(y_true=y_label, y_pred=pred_classes, classes=test_data.class_names, figsize=(100, 100))

Creating classification report¶

In [ ]:
from sklearn.metrics import classification_report
print(classification_report(y_true=y_label, y_pred=pred_classes))
              precision    recall  f1-score   support

           0       0.32      0.23      0.27       250
           1       0.52      0.76      0.62       250
           2       0.62      0.60      0.61       250
           3       0.92      0.48      0.63       250
           4       0.58      0.44      0.50       250
           5       0.48      0.39      0.43       250
           6       0.81      0.73      0.77       250
           7       0.84      0.77      0.80       250
           8       0.28      0.53      0.37       250
           9       0.39      0.70      0.50       250
          10       0.61      0.37      0.46       250
          11       0.84      0.54      0.66       250
          12       0.65      0.59      0.62       250
          13       0.55      0.58      0.56       250
          14       0.48      0.62      0.54       250
          15       0.49      0.28      0.36       250
          16       0.77      0.50      0.60       250
          17       0.36      0.49      0.41       250
          18       0.45      0.55      0.50       250
          19       0.62      0.66      0.64       250
          20       0.77      0.74      0.76       250
          21       0.52      0.61      0.56       250
          22       0.38      0.38      0.38       250
          23       0.72      0.74      0.73       250
          24       0.77      0.67      0.72       250
          25       0.74      0.68      0.71       250
          26       0.39      0.36      0.38       250
          27       0.78      0.78      0.78       250
          28       0.76      0.69      0.72       250
          29       0.78      0.70      0.74       250
          30       0.88      0.62      0.73       250
          31       0.72      0.80      0.76       250
          32       0.80      0.85      0.82       250
          33       0.94      0.98      0.96       250
          34       0.67      0.75      0.71       250
          35       0.70      0.65      0.67       250
          36       0.51      0.42      0.46       250
          37       0.27      0.58      0.37       250
          38       0.75      0.68      0.71       250
          39       0.53      0.22      0.31       250
          40       0.70      0.77      0.74       250
          41       0.64      0.68      0.66       250
          42       0.64      0.45      0.53       250
          43       0.80      0.54      0.65       250
          44       0.78      0.46      0.58       250
          45       0.86      0.79      0.82       250
          46       0.64      0.63      0.64       250
          47       0.42      0.55      0.48       250
          48       0.53      0.73      0.61       250
          49       0.41      0.49      0.45       250
          50       0.59      0.25      0.35       250
          51       0.75      0.88      0.81       250
          52       0.66      0.64      0.65       250
          53       0.62      0.72      0.67       250
          54       0.82      0.84      0.83       250
          55       0.79      0.81      0.80       250
          56       0.37      0.40      0.39       250
          57       0.43      0.49      0.46       250
          58       0.60      0.60      0.60       250
          59       0.64      0.35      0.45       250
          60       0.74      0.65      0.69       250
          61       0.76      0.62      0.69       250
          62       0.45      0.67      0.54       250
          63       0.93      0.90      0.91       250
          64       0.66      0.90      0.76       250
          65       0.85      0.80      0.82       250
          66       0.70      0.35      0.47       250
          67       0.40      0.52      0.45       250
          68       0.83      0.84      0.84       250
          69       0.89      0.83      0.86       250
          70       0.81      0.82      0.82       250
          71       0.85      0.38      0.52       250
          72       0.65      0.64      0.65       250
          73       0.37      0.61      0.46       250
          74       0.85      0.48      0.61       250
          75       0.85      0.86      0.85       250
          76       0.72      0.89      0.80       250
          77       0.30      0.32      0.31       250
          78       0.71      0.72      0.71       250
          79       0.54      0.73      0.62       250
          80       0.57      0.70      0.63       250
          81       0.52      0.78      0.62       250
          82       0.36      0.27      0.31       250
          83       0.57      0.78      0.66       250
          84       0.45      0.41      0.43       250
          85       0.57      0.64      0.60       250
          86       0.83      0.85      0.84       250
          87       0.41      0.39      0.40       250
          88       0.78      0.80      0.79       250
          89       0.43      0.52      0.47       250
          90       0.89      0.62      0.73       250
          91       0.75      0.90      0.82       250
          92       0.76      0.57      0.65       250
          93       0.33      0.22      0.27       250
          94       0.68      0.40      0.50       250
          95       0.81      0.54      0.65       250
          96       0.56      0.44      0.49       250
          97       0.70      0.64      0.67       250
          98       0.43      0.54      0.48       250
          99       0.30      0.48      0.37       250
         100       0.77      0.59      0.67       250

    accuracy                           0.61     25250
   macro avg       0.63      0.61      0.61     25250
weighted avg       0.63      0.61      0.61     25250

In [ ]:
# Get a dictionary of classification report
classification_report_dict = classification_report(y_true=y_label, y_pred=pred_classes, output_dict=True)
In [ ]:
class_names = test_data.class_names

Ploting F1 scores¶

In [ ]:
# Create empty dictionary
class_f1_scores = {}

# Loop through classification report
for k, v in classification_report_dict.items():
    if k == "accuracy": # Stop once we get to accuracy key
        break
    else:
        # Add class names and F1 scores o new dictionaty
        class_f1_scores[class_names[int(k)]] = v["f1-score"]
class_f1_scores
Out[ ]:
{'apple_pie': 0.2703962703962704,
 'baby_back_ribs': 0.6161290322580644,
 'baklava': 0.6138211382113822,
 'beef_carpaccio': 0.6279683377308708,
 'beef_tartare': 0.4965831435079727,
 'beet_salad': 0.43171806167400884,
 'beignets': 0.76890756302521,
 'bibimbap': 0.8024948024948025,
 'bread_pudding': 0.3658872077028886,
 'breakfast_burrito': 0.5035765379113019,
 'bruschetta': 0.45885286783042395,
 'caesar_salad': 0.6601941747572816,
 'cannoli': 0.6205450733752621,
 'caprese_salad': 0.5620155038759689,
 'carrot_cake': 0.5445026178010471,
 'ceviche': 0.35714285714285715,
 'cheese_plate': 0.6019417475728156,
 'cheesecake': 0.4141414141414142,
 'chicken_curry': 0.49640287769784175,
 'chicken_quesadilla': 0.6434108527131783,
 'chicken_wings': 0.7551020408163266,
 'chocolate_cake': 0.5598526703499079,
 'chocolate_mousse': 0.376,
 'churros': 0.7272727272727273,
 'clam_chowder': 0.7194860813704498,
 'club_sandwich': 0.708595387840671,
 'crab_cakes': 0.37837837837837834,
 'creme_brulee': 0.781563126252505,
 'croque_madame': 0.7238493723849372,
 'cup_cakes': 0.7410526315789474,
 'deviled_eggs': 0.7259953161592505,
 'donuts': 0.7566539923954372,
 'dumplings': 0.8239845261121858,
 'edamame': 0.9609375000000001,
 'eggs_benedict': 0.7069943289224954,
 'escargots': 0.6749482401656315,
 'falafel': 0.46389496717724293,
 'filet_mignon': 0.3697047496790757,
 'fish_and_chips': 0.7100840336134454,
 'foie_gras': 0.3068181818181818,
 'french_fries': 0.7352380952380952,
 'french_onion_soup': 0.6627906976744187,
 'french_toast': 0.5283018867924527,
 'fried_calamari': 0.645933014354067,
 'fried_rice': 0.577889447236181,
 'frozen_yogurt': 0.8232848232848233,
 'garlic_bread': 0.6370967741935485,
 'gnocchi': 0.47735191637630664,
 'greek_salad': 0.6127946127946128,
 'grilled_cheese_sandwich': 0.4468864468864469,
 'grilled_salmon': 0.3539325842696629,
 'guacamole': 0.8118081180811809,
 'gyoza': 0.6504065040650406,
 'hamburger': 0.6691449814126393,
 'hot_and_sour_soup': 0.8293650793650793,
 'hot_dog': 0.7999999999999999,
 'huevos_rancheros': 0.3877159309021113,
 'hummus': 0.4577861163227017,
 'ice_cream': 0.6024096385542169,
 'lasagna': 0.45360824742268036,
 'lobster_bisque': 0.6936170212765959,
 'lobster_roll_sandwich': 0.6857142857142857,
 'macaroni_and_cheese': 0.5410628019323671,
 'macarons': 0.9142857142857143,
 'miso_soup': 0.7640067911714771,
 'mussels': 0.822314049586777,
 'nachos': 0.46933333333333327,
 'omelette': 0.4509466437177281,
 'onion_rings': 0.8356435643564355,
 'oysters': 0.8577319587628867,
 'pad_thai': 0.8158415841584158,
 'paella': 0.5248618784530387,
 'pancakes': 0.6491935483870968,
 'panna_cotta': 0.4613050075872534,
 'peking_duck': 0.6122448979591836,
 'pho': 0.8542914171656687,
 'pizza': 0.7950089126559716,
 'pork_chop': 0.3079922027290448,
 'poutine': 0.7142857142857143,
 'prime_rib': 0.6179966044142615,
 'pulled_pork_sandwich': 0.6261180679785331,
 'ramen': 0.6217948717948717,
 'ravioli': 0.30979498861047833,
 'red_velvet_cake': 0.6598984771573604,
 'risotto': 0.4291666666666667,
 'samosa': 0.6003752345215759,
 'sashimi': 0.8379446640316205,
 'scallops': 0.40163934426229503,
 'seaweed_salad': 0.7913385826771654,
 'shrimp_and_grits': 0.4716636197440585,
 'spaghetti_bolognese': 0.7341176470588235,
 'spaghetti_carbonara': 0.8173598553345389,
 'spring_rolls': 0.6484018264840182,
 'steak': 0.2660332541567696,
 'strawberry_shortcake': 0.50126582278481,
 'sushi': 0.6457831325301205,
 'tacos': 0.49217002237136465,
 'takoyaki': 0.6708595387840671,
 'tiramisu': 0.48042704626334526,
 'tuna_tartare': 0.3726708074534162,
 'waffles': 0.6681715575620767}
In [ ]:
# turn f1 scores into dataframe for visualization
import pandas as pd

f1_scores = pd.DataFrame({"class_names": list(class_f1_scores.keys()),
                         "f1-score": list(class_f1_scores.values())}).sort_values("f1-score", ascending=False)
In [ ]:
f1_scores[:10]
Out[ ]:
class_names f1-score
33 edamame 0.960938
63 macarons 0.914286
69 oysters 0.857732
75 pho 0.854291
86 sashimi 0.837945
68 onion_rings 0.835644
54 hot_and_sour_soup 0.829365
32 dumplings 0.823985
45 frozen_yogurt 0.823285
65 mussels 0.822314
In [ ]:
import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(12, 25))
scores = ax.barh(range(len(f1_scores)), f1_scores["f1-score"].values)
ax.set_yticks(range(len(f1_scores)))
ax.set_yticklabels(f1_scores["class_names"])
ax.set_xlabel("F1-score")
ax.set_title("F1-score for 101 food classes")
ax.bar_label(scores, fmt="%.2f")
ax.invert_yaxis();

Visualizing predictions on test images¶

Steps:

  • Read in a target inmage filepath using tf.io.read_file()
  • turn the image into a tensor using tf.io.decode_image()
  • Resize the image tensor to be the same size as images our model has trained on using tf.image.resize()
  • Scale the image to get all of the pizel values between 0 & 1
In [ ]:
# Create a function to load and prepare images
def load_and_prep_image(filename, img_size=224, scale=True):
    """
    reads in an image form filename, turns it into a tensor and reshapes into specified shape
    
    Args:
        filename (str): path to target image
        img_shape (int): height/width dimentions of target image size
        scale (bool): scale pixel values from 0-255 to  0-1 or not
        
    Returns:
        Image tensor of shape (img_shape, img_shape, 3)
    """
    # Read in the image
    img = tf.io.read_file(filename=filename)
    
    # Decode image into tensor
    img = tf.io.decode_image(img, channels=3)
    
    # Resize the image
    img = tf.image.resize(img, [img_size, img_size])
    
    # Scaling
    if scale:
        # Rescale the image
        return img/255.
    else:
        return img
In [ ]:
# visualize test images

# Make preds on series of random images
import os
import random

plt.figure(figsize=(17, 10))
for i in range(3):
    # Choose a random image from random class
    class_name = random.choice(class_names)
    filename = random.choice(os.listdir(test_dir + "/" + class_name))
    filepath = test_dir + "/" + class_name + "/" + filename
    
    # Load the images and make predictions
    img = load_and_prep_image(filepath, scale=False)
    img_expanded = tf.expand_dims(img, axis=0)
    # pred_prob = model.predict(img_expanded) # get perdiction probability array
    pred_prob = loaded_model.predict(img_expanded) # get perdiction probability array
    pred_class = class_names[pred_prob.argmax()] # get highest prediction probability index
    
    # Plot the images
    plt.subplot(1, 3, i+1)
    plt.imshow(img/255)
    if class_name == pred_class: # if predicted class matches, truth class, make text green
        title_color = "g"
    else:
        title_color = "r"
    plt.title(f"Actual class: {class_name}, pred: {pred_class}, prob: {pred_prob.max():.2f}", c=title_color)
    plt.axis(False);
    
1/1 [==============================] - 1s 1s/step
1/1 [==============================] - 0s 25ms/step
1/1 [==============================] - 0s 25ms/step

Finding the most wrong predictions¶

In [ ]:
# get all of the files from test_data
filepaths = []
for filepath in test_data.list_files("101_food_classes_10_percent/test/*/*.jpg", shuffle=False):
    filepaths.append(filepath.numpy())
    
filepaths[:10]
Out[ ]:
[b'101_food_classes_10_percent/test/apple_pie/1011328.jpg',
 b'101_food_classes_10_percent/test/apple_pie/101251.jpg',
 b'101_food_classes_10_percent/test/apple_pie/1034399.jpg',
 b'101_food_classes_10_percent/test/apple_pie/103801.jpg',
 b'101_food_classes_10_percent/test/apple_pie/1038694.jpg',
 b'101_food_classes_10_percent/test/apple_pie/1047447.jpg',
 b'101_food_classes_10_percent/test/apple_pie/1068632.jpg',
 b'101_food_classes_10_percent/test/apple_pie/110043.jpg',
 b'101_food_classes_10_percent/test/apple_pie/1106961.jpg',
 b'101_food_classes_10_percent/test/apple_pie/1113017.jpg']
In [ ]:
# Create a dataframe of different parameters
pred_df = pd.DataFrame({"img_path": filepaths,
                       "y_true": y_label,
                       "y_pred": pred_classes,
                       "pred_conf": preds_probs.max(axis=1), # Get the maximum prediction probability value
                       "y_true_classname": [class_names[i] for i in y_label],
                       "y_pred_classname": [class_names[i] for i in pred_classes]})
pred_df
Out[ ]:
img_path y_true y_pred pred_conf y_true_classname y_pred_classname
0 b'101_food_classes_10_percent/test/apple_pie/1... 0 0 0.213262 apple_pie apple_pie
1 b'101_food_classes_10_percent/test/apple_pie/1... 0 0 0.828701 apple_pie apple_pie
2 b'101_food_classes_10_percent/test/apple_pie/1... 0 0 0.716567 apple_pie apple_pie
3 b'101_food_classes_10_percent/test/apple_pie/1... 0 8 0.197051 apple_pie bread_pudding
4 b'101_food_classes_10_percent/test/apple_pie/1... 0 8 0.460519 apple_pie bread_pudding
... ... ... ... ... ... ...
25245 b'101_food_classes_10_percent/test/waffles/942... 100 100 0.933011 waffles waffles
25246 b'101_food_classes_10_percent/test/waffles/954... 100 16 0.576720 waffles cheese_plate
25247 b'101_food_classes_10_percent/test/waffles/961... 100 100 0.369479 waffles waffles
25248 b'101_food_classes_10_percent/test/waffles/970... 100 94 0.279318 waffles strawberry_shortcake
25249 b'101_food_classes_10_percent/test/waffles/971... 100 100 0.441311 waffles waffles

25250 rows × 6 columns

In [ ]:
# Find which predictions are wrong
pred_df["pred_correct"] = pred_df["y_true"] == pred_df["y_pred"]
pred_df.head()
Out[ ]:
img_path y_true y_pred pred_conf y_true_classname y_pred_classname pred_correct
0 b'101_food_classes_10_percent/test/apple_pie/1... 0 0 0.213262 apple_pie apple_pie True
1 b'101_food_classes_10_percent/test/apple_pie/1... 0 0 0.828701 apple_pie apple_pie True
2 b'101_food_classes_10_percent/test/apple_pie/1... 0 0 0.716567 apple_pie apple_pie True
3 b'101_food_classes_10_percent/test/apple_pie/1... 0 8 0.197051 apple_pie bread_pudding False
4 b'101_food_classes_10_percent/test/apple_pie/1... 0 8 0.460519 apple_pie bread_pudding False
In [ ]:
# Sort dataframe to have most wrong predictions at top
top_100_wrong = pred_df[pred_df["pred_correct"] == False].sort_values("pred_conf", ascending=False)[:100]
top_100_wrong.head(20)
Out[ ]:
img_path y_true y_pred pred_conf y_true_classname y_pred_classname pred_correct
23797 b'101_food_classes_10_percent/test/sushi/16593... 95 86 0.999345 sushi sashimi False
18001 b'101_food_classes_10_percent/test/pancakes/10... 72 67 0.998549 pancakes omelette False
191 b'101_food_classes_10_percent/test/apple_pie/3... 0 31 0.997635 apple_pie donuts False
17855 b'101_food_classes_10_percent/test/paella/2314... 71 65 0.997257 paella mussels False
8794 b'101_food_classes_10_percent/test/escargots/1... 35 41 0.996961 escargots french_onion_soup False
10847 b'101_food_classes_10_percent/test/fried_calam... 43 68 0.996513 fried_calamari onion_rings False
15142 b'101_food_classes_10_percent/test/lobster_bis... 60 64 0.996350 lobster_bisque miso_soup False
21810 b'101_food_classes_10_percent/test/scallops/17... 87 29 0.996283 scallops cup_cakes False
7194 b'101_food_classes_10_percent/test/croque_mada... 28 7 0.994985 croque_madame bibimbap False
23631 b'101_food_classes_10_percent/test/strawberry_... 94 83 0.993791 strawberry_shortcake red_velvet_cake False
11642 b'101_food_classes_10_percent/test/garlic_brea... 46 10 0.993379 garlic_bread bruschetta False
5849 b'101_food_classes_10_percent/test/churros/254... 23 68 0.992862 churros onion_rings False
10013 b'101_food_classes_10_percent/test/french_frie... 40 43 0.992745 french_fries fried_calamari False
10880 b'101_food_classes_10_percent/test/fried_calam... 43 68 0.992740 fried_calamari onion_rings False
10854 b'101_food_classes_10_percent/test/fried_calam... 43 68 0.991826 fried_calamari onion_rings False
722 b'101_food_classes_10_percent/test/baklava/640... 2 85 0.991008 baklava samosa False
14691 b'101_food_classes_10_percent/test/ice_cream/3... 58 79 0.990902 ice_cream prime_rib False
12487 b'101_food_classes_10_percent/test/grilled_che... 49 25 0.990754 grilled_cheese_sandwich club_sandwich False
15359 b'101_food_classes_10_percent/test/lobster_rol... 61 53 0.990743 lobster_roll_sandwich hamburger False
5114 b'101_food_classes_10_percent/test/chicken_win... 20 38 0.990371 chicken_wings fish_and_chips False
In [ ]:
# Visualize wrong predictions
images_to_view = 9
start_index = 10
plt.figure(figsize=(30, 20))
for i, row in enumerate(top_100_wrong[start_index:start_index + images_to_view].itertuples()):
    plt.subplot(3, 3, i+1)
    img = load_and_prep_image(row[1], scale=False)
    _, _, _, _, pred_prob, y_true_classname, y_pred_classname, _ = row # skipping few values
    plt.imshow(img/255)
    plt.title(f"actual: {y_true_classname}, pred: {y_pred_classname}, prob: {pred_prob}")
    plt.axis(False);
    

Test model on custom images¶

In [ ]:
!python -m wget https://storage.googleapis.com/ztm_tf_course/food_vision/custom_food_images.zip
--2023-03-13 22:28:29--  https://storage.googleapis.com/ztm_tf_course/food_vision/custom_food_images.zip
Resolving storage.googleapis.com (storage.googleapis.com)... 142.251.41.80, 172.217.165.16, 142.251.32.80, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|142.251.41.80|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 13192985 (13M) [application/zip]
Saving to: ‘custom_food_images.zip’

custom_food_images. 100%[===================>]  12.58M  29.7MB/s    in 0.4s    

2023-03-13 22:28:30 (29.7 MB/s) - ‘custom_food_images.zip’ saved [13192985/13192985]

In [ ]:
unzip_data("custom_food_images.zip")
walk_through_dir("custom_food_images")
There are 0 directories and 6 images in 'custom_food_images'.
In [ ]:
# Get custom food images filepaths
custom_food_images = ["custom_food_images/" + img_path for img_path in os.listdir("custom_food_images")]
custom_food_images
Out[ ]:
['custom_food_images/chicken_wings.jpeg',
 'custom_food_images/ramen.jpeg',
 'custom_food_images/sushi.jpeg',
 'custom_food_images/hamburger.jpeg',
 'custom_food_images/pizza-dad.jpeg',
 'custom_food_images/steak.jpeg']
In [ ]:
# make predictions and plot custom food images
for img in custom_food_images:
    img = load_and_prep_image(img, scale=False)
    pred_prob = model.predict(tf.expand_dims(img, axis=0))
    pred_class = class_names[pred_prob.argmax()]
    
    # Plot the information
    plt.figure()
    plt.imshow(img/255.)
    plt.title(f"pred: {pred_class}, prob: {pred_prob.max():.2f}")
    plt.axis(False);
1/1 [==============================] - 1s 792ms/step
1/1 [==============================] - 0s 28ms/step
1/1 [==============================] - 0s 29ms/step
1/1 [==============================] - 0s 24ms/step
1/1 [==============================] - 0s 23ms/step
1/1 [==============================] - 0s 23ms/step
In [ ]: